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Abstract 

This paper develops numerical methods for finding optimal dividend pay-out and 
reinsurance policies. A generalized singular control formulation of surplus and dis- 
counted payoff function are introduced, where the surplus is modeled by a regime- 
switching process subject to both regular and singular controls. To approximate the 
value function and optimal controls, Markov chain approximation techniques are used 
to construct a discrete-time controlled Markov chain with two components. The proofs 
of the convergence of the approximation sequence to the surplus process and the value 
function are given. Examples of proportional and excess-of-loss reinsurance are pre- 
sented to illustrate the applicability of the numerical methods. 
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1 Introduction 



To design optimal risk controls and and dividend payout strategies for a financial corpo- 
ration h as drawn increas ing attention since the introduction of the classical collective risk 
model in iLundberg] ( 1l903l ). where the probability of ruin was considered as a measure of risk. 
Realizing that the s urplus reaching a rbitrarily high and exceeding any finite level are not 
realistic in practice, iDe Finettil ( 119571 ) proposed an dividend optimization problem. Instead 
of considering the safety aspect (ruin probability), aiming at maximizing the expected dis- 
counted total dividends until lifetime ruin by assuming the surplus process follows a simple 
random walk, he showed that the optimal dividend strategy is a barrier strategy. Since then, 
many researchers have analyzed this problem under more reali stic assumptions and extende d 
its range of applicat i ons. Some recent work can be found in lAsmussen and Taksarl (119971 ): 
ChouUi et al.l (l200l[ ): iGerber and Shiul (|2004[ ) and references therein. To protect insurance 
companies against the impact of claim volatilities, reinsurance is a standard tool with the 
goal of reducing and eliminating risk. The primary insurance carrier pays the reinsurance 
company a certain part of the premiums. In return, the reinsurance company is obliged 
to share the risk of large claims. Proportional reinsurance is one type of reinsurance pol- 
icy. Within this scheme, the reinsurance company covers a fixed percentage of losses. The 
other type of reinsurance policy is nonproportional reinsurance. The most common non- 
proportional reinsurance policy is the so-called excess-of-loss reinsurance, within which the 
cedent (primary insurance carrier) will pay all of the claims up to a pre-given level of amount 
(t ermed retention level) . The comparison of these two types of reinsurance can be found 
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Asmusen et al 



(|2000[ ). In this paper, we consider both of these reinsurance policies and 
provide the numerical solutions of the corresponding Markovian regime-switching models. 

Let u{t) be an exogenous retention level, which is a control chosen by the insurance 
company representing the reinsurance policy. In a Cremer-Lundberg model, claims arrive 
according a Poisson process with rate /3. Let Yi be the size of the ith claim. The Fj's are 
independent and identically distributed (i.i.d.) random variables. Let Y^^ be the the fraction 
of the claims hold by the cedent. The insurer selects the time and the amount of dividends 
to be paid out to the policyholders. Let X{t) denote the controlled surplus of an insurance 
company at time t > 0. Throughout this paper, we only consider cheap reinsurance, where 
the safety loading for the reinsurer is the same as that for the cedent. The numerical scheme 
and the convergence proofs are also applicable to more general reinsurance problems. By 
using the techniques of diffusion approximation applied to the Cremer-Lundberg model, the 
surplus process satisfies 




/3E[Yr]dt + ,/mWm)dw{t), 



'1.1] 



X, 



where w{t) is a standard Brownian motion. In the case of proportional reinsurance, Y^^ = uYi. 
Thus, following (II. ip . the surplus is given by 



idXit) = /3u{t)E[Yi]dt + u{t)y/(3E[Y^^]dw{t), 
[X{0~)=x, 



:i.2) 
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In the case of excess-of-loss reinsurance, Y^^ = Yi Au with the retention level u. We have 

^[r"] = / F{x)dx, = / 2xF{x)dx (1.3) 



JO JO 

where F{x) = P{Yi > x). The stochastic differential equation of the surplus process follows 



Mt) _ 

dX{t) = P F{x)dxdt + 
Jo 

X(O') =x. 



u{t) _ 

P I F{x)dx ^dw{t), 





;i-4) 



A common choice of the pay off is to maximize the to tal e xpected di s count ed value of all 
dividends until lifetime ruin; see lGerber and Shiul teooeh and I Jin et all fcoilh . Let 



T := inf {t > : X{t) ^ G} 



;i.5) 



be the ruin time, where G = (0, oo) is the domain of the surplus. Denote by r > the 
discounting factor, and by Z{t) the total dividends paid out up to time t. Our goal is to 
maximize ^ 

E [ e-'^'dZit). (1.6) 
Jo 

Some "bequest" func tions and mor e corn plicated utility functions are added to the payoff 
functions in the work iBrownd (119951 . 119971 ). In this pap er, we treat payoff fun ctions that are 
more general and complex than those given in fll.6p or iBrownd (119951 . 119971 ) ; our proposed 
numerical methods are easily implementable. 

A dividend strategy Z(-) is an J^^-adapted process {Z{t) : t > 0} corresponding to the 
accumulated amount of dividends paid up to time t such that Z{t) is a nonnegative and non- 
decreasing stochastic process that is right continuous with left limits. Throughout the paper, 
we use the convention that Z{0~) = 0. In general, a dividend process is not necessarily ab- 
solutely continuous. In fact, dividends are not usually paid out continuously in practice. For 
instance, insurance companies may distribute dividends on discrete time intervals resulting 
in unbounded payment rate. In such a scenario, the surplus level changes drastically on a 
dividend payday. Thus abrupt or discontinuous changes occur due to "singular" dividend 
distribution policy. Together with proportional or excess-of-loss reinsurance policy, this gives 
rise to a mixed regular-singular stochastic control problem. 

Empirical studies indicate in particular that traditional surplus models fail to capture 
more extreme price movements. To better reflect reality, much effort has been devot ed to pro- 
ducing better models. One of the recent trends is to use regime- switching models. iHamilton 
(Il989l ) introduced a regime-s witching ti r ne ser i es model. Recent wo rk o n risk models an d 
related issues can be found in lAsmussenl (119891 ) ; lYang and YinI (120041 ) . In IWei et al.l ( l2010l ) , 
the optimal dividend and proportional reinsurance strategy under utility criteria were stud- 
ied for the regime- switching c ompound Poisson model by using the methods of the classical 



and impulse control theory. ISotomayor and Cadenillad (120111 ) obtained optimal dividend 



strategies under a regime-switching diffusion m odel. A comp r ehens ive study of switching 
diffusions with "state-dependent" switching is in lYin and Zhd (120101 ). 

In this work, we model the surplus process by a regime-switching diffusion; reinsurance 
and dividend payment policies are introduced as regular and singular stochastic controls. 
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The goal is to maximize the expected total discounted payoff until ruin; see (12. 2p for details. 
The model we consider appears to be more versatile and realistic than the classical compound 
Poisson or diffusion models. To find the optimal reinsurance and dividend pay-out strate- 
gies, one usually solves a so-called Hamilton- Jacobi-Bellman (HJB) equation. However, in 
our work, thanks to regime switching and the mixed regular and singular control formula- 
tion, the HJB equation is in fact a coupled system of nonlinear quasi- variational inequalities 
(QVIs). A closed-form solution is virtually impossible to obtain. A viable alternative is to 
employ numerical approxi mations. In this work, we ad apt the Markov chain approximation 
methodology developed by lKushner and DupuisI (120011 ). To the best of our knowledge, nu- 
merical methods for singular controls of regime-switching diffusions have not been studied in 
the literature to date. Even for sing ular controlled diffusi o ns wi thout regime switching, the 
relate d results are relatively scarce; iBudhiraja and Rossi (120071 ) and iKushner and Martins 
(119911 ) are the only papers that carry out a convergence analysis using weak convergence 
and relaxed control formulation of numerical schemes for singular control problems in the 
setting of Ito diffusions. We focus on developing numerical methods that are applicable 
to mixed regular and singular controls for regime-switching models. Although the primary 
motivation stems from insurance risk controls, the techniques and the algorithms suggested 
appear to be applicable to other singular control problems. It is also worth mentioning 
that the Markov chain approximation method requires little regularity of the value function 
and/or analytic properties of the associated systems of HJB equations and/or QVIs. The 
numerical implementation can be done using either value iterations or policy iterations. 

The rest of the paper is organized as follows. A generalized formulation of optimal risk 
control and dividend policies and assumptions are presented in Section |2l The two most 
common types of reinsurance strategy (proportional reinsurance and excess-of-loss reinsur- 
ance) are covered in our study. Section [3] deals with the numerical algorithm of Markov 
chain approximation method. The regular control and the singular control are well ap- 
proximated by the approximating Markov chain and the dynamic programming equation 
are presented. Section H] deals with the convergence of the approximation scheme. The 
technique of "rescaling time" is introduced and the convergence theorems are proved. Two 
classes of numerical examples are provided in Section |5] to illustrate the performance of the 
approximation method. Finally, some additional remarks are provided in Section |6l 



2 Formulation 

In this section, we introduce a dynamic system to describe the surplus processes with rein- 
surance and dividend payout strategies with Markov regime switching. Let X{t) denote the 
controlled surplus of an insurance company at time t >0. Denote by u{t) and Z{t) the dy- 
namic reinsurance policy at time t and the total dividend paid out up to time t, respectively. 
Assume the evolution of X{t), subject to reinsurance and dividend payments, follows a one- 
dimensional temporal homogeneous controlled regime- switching diffusion on an unbounded 
domain G = (0, oo): 

dX{t) = b{X{t), a{t),u{t))dt + (T{X{t), a{t),u{t))dW{t) - dZ{t), 
X{0~)=xeG, a{0-) = iEM, 
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where u is the regular control and Z is the singular control. Throughout the paper we 
use the convention that Z{0~) = 0. The jump size of Z at time t > is denoted by 
AZ{t) := Z{t) — Z(t~), and Z'^(t) := Z(t) — J2o<s<t^^i^) denotes the continuous part of 
Z. Also note that AX{t) := X{t) - X{t-) = -AZ(t) for any t > 0. 

Denote by r > the discounting factor. For suitable functions / and c and an arbitrary 
admissible pair n = {u, Z), the expected discounted payoff is 



J(x, i, tt) 



-rt 



[/(X(t), a{t),u{t))dt + c(X(r ), a{t-))dZ{t)] 



(2.2) 



The pair vr = {u, Z) is said to be admissible if u and Z satisfy 

(i) u{t) and Z{t) are nonnegative for any t > 0, 

(ii) Z is cadlag and nondecreasing, 

(iii) X(t) > 0, for any t < t, where r is the ruin time defined in (11.51) . 

(iv) both u and Z are adapted to J-i := a {W{s), a{s), < s <t} augmented by the P-nuU 
sets, and 

(iv) J{x, £,tt) < oo for any {x,£) & G x Ai and admissible pair vr = {u, Z), where J is the 
functional defined in (12.21) . 

Suppose that A is the collection of all admissible pairs, and U is the collection of possible 
retention levels u{t). Throughout the paper, we assume that f/ is a given compact set, and 
that for each i E Ai, c{x,i) > c{y,i) for all < x < y. That is, th e util ity funct i on for the 
dividend is non-decreasing, see examples in iGerber and Shiul (120051 ) and lAlvarezI (120001 ) . In 
addition, c{X(t),£) = f{X{t),i,u) = when t > r. Define the value function as 



V{x, £) := sup J(x, i, vr). 



(2.3) 



If the value function V defined in (|2.3p is suffi ciently smooth, by applying the dynamic 
programming principle ( Fleming and Soner ( 20061 )). we conclude formerly that V satisfies 
the following coupled system of quasi variational inequalities (QVIs): 

max {H{x, £, V'{x, i), V"{x, £)) + Q{x)V{x, ■){£) - rV{x, £), c(x, £) - V'{x, £)} = 0, (2.4) 

for all {t,x,£) G [0, r) x G x A^, with boundary condition 

y(0,£) = 0, WeM, (2.5) 

where for any {x,£,p) G xM" x x R", 

H{x,£,p) := sup {f{x,£,u) + p- b{x,£,u) + l-a'^{x,£,u)}, 

u€U 2 

Qix)V{x, ■){£) := J2 Qd^Wi^, - V{x, £)], 



and V and V" denote the first and the second partial derivatives of V with respect to x. 
Note that couphng i n (|2.4p is due to tfi e terrn Q(x)V(t,x, wh ic h is no t cont ained in 



the usual QVI (as in iFleming and Sonerl ( 120061 ): iMa and Yongi f ll999l ): IPhaml f l2009l )). 



Nevertheless, t he va lue function V is not necessarily smooth. In fact, there are examples 
teavraktar et al.l fl201l[ )) where the value function is not even continuous. In our work, both 
the ruin time r and the reinsurance u and the dividend Z policies may depend on the initial 
surplus level, which leads to nonsmooth vale function. Moreover, (12. 4p is a coupled system of 
nonlinear differential equations. A closed-form solution to (12. 4p is by and large impossible. 
Therefore in this work, we propose a numerical scheme to approximate the value function 
as well as optimal reinsurance and dividend payment pohcies. 



3 Numerical Algorithm 

Our goal is to design a numerical scheme to approximate value function V in (12. 3p . As a 
standing assumption, we assume V{-) is continuous with respect to x. In this section we 
construct a locally consistent Markov chain approximation for the mixed regular-singular 
control model with regime-switching. The discrete-time and finite-state controlled Markov 
chain is so defined that it is locally consistent with (12. ip . Note that th e state of the process 



has tw o components x and a. Hence in order to use the methodology in lKushner and Dupuis 



( 120011 ). our approximating Markov chain must have two components: one component delin- 
eates the diffusive behavior whereas the other keeps track of the regimes. Let h > he a 
discretization parameter. Define Lh = {x : x = kh, k = 0, ±1, ±2, . . . } and Sh = L^H Gh, 
where Gh = {0,B + h) and B is an upper bound introduced for numerical computation pur- 
pose. Moreover, assume without loss of generality that the boundary point B is an integer 
multiple of h. Let {(^^, a(^), n < oo} be a controlled discrete-time Markov chain on S"/! x 
and denote by p'^{{x, i), {y, l)\h'^) the transition probability from a state {x,£) to another 
state {y, l) under the control n^. We need to define so that the chain's evolution well 
approximates the local behavior of the controlled regime-switching diffusion (12. ip . At any 
discrete time n, we can either exercise a regular control, a singular control or a reflection 
step. That is, if we put A,^^ = ^^^^ — then 

= Ae n {regular control step at n} n-' {singular control step at n} + ^{reflection step at n} ■ (3.1) 

The chain and the control will be chosen so that there is exactly one term in (13. ip is nonzero. 
Denote by {/^ : n = 0, 1, . . . } a sequence of control actions, where = 0, 1 or 2, if we 
exercise a singular control, regular control, or reflection at time n, respectively. 

If = 1, then we denote by C f/ the random variable that is the regular control 
action for the chain at time n. Let At^{-, - , ■) > be the interpolation interval on ShxAixU. 
Assume inf^.^^^^ Ai:'*(x, m) > for each h> and limh^QSup^ ^ ^At^{x,i,u) — > 0. 

Let E^'^'^, Var"'^'j^ and P^f^ denote the conditional expectation, variance, and marginal 
probability given {C^ji^,a^,u^,ll^,k < n,^'^ = x,a^ = = l,u^ = u}, respectively. The 
sequence {(^^,a^)} is said to be locally consistent, if it satisfies 
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P:tn{<+i = 4 = g£.(.x)At''(a;,£,M) + o{Kt\x,^,u)), for . ^ 
^.tn{«n+i = n = (1 + to(x))At^(x,^,«) + o(At'^(a:, 
sup |A^^| ^0 as /i ^ 0. 

If = 0, then we denote by Az^ the random variable that is the singular control action 
for the chain at time n if e [0, B]. Note that A^^ = -A;^^ = -h. If = 2, or = S + 
reflection step is exerted definitely. Dividend is paid out to lower the surplus level. Moreover, 
we require reflection takes the state from B + h to B. That is, if we denote by A^f^ the 
random variable that is the reflection action for the chain at time n, then A,^^ = —Agn = —h. 

The singular control can be seen as a combination of "inside" part (/^ = 0) and "bound- 
ary" part (/j^ = 2). Also we require the singular control and reflection to be "impulsive" or 
"instantaneous." In other words, the interpolation interval on Sh x M. x U x {0, 1, 2} is 

At^(,T, £, u, i) = At^{x, i, M)/{i=i}, for any {x, i,u,i) e Sh x M x U x {0, 1, 2} . (3.2) 

Denote by :— {tt^, n > 0} the sequence of control actions, where 

The sequence is said to be admissible if tt^ is a {(Co; c^o); • • • > (Cn; '^n); '^o ; • • • ; '^n-i}~ 
adapted and for any E e B{Sh x M.), we have 

P {(Ci, "n+i) e E\a{{eo, <),..., {tn, ■■■.<]]= P\{it <l E\n% 

and 

p <i) = {B. I (e, «n) -{B^Kit). o{{ii, co^), . . . , (e, «::), 4, . . . , tt^^}} = i. 

Put 

n-1 

^^=0, ^^= J]At''(4\a^<4^), and n'^(i) := max{n : < t} . 

fe=0 

Then the piecewise constant interpolations, denoted by (^'^(•), «'*(•)), «'*(•), 5'''(-)) a-i^d -^'*(')) 
are naturally defined as 

(3.3) 

for t e [^n5^n+i)- Let rjh := inf {n : G dG}. Then the first exit time of ^'^ from G is 
r'* — ■'-'^^ ' '^o) = £) G X and be an admissible control. The cost function 
for the controlled Markov chain is defined as 

jUx, e, tt'^) = £^ E [/(^^ «^ ^^)^^^ + 4)^4], (3.4) 

fe=l 
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which is analogous to (12. 2p thanks to the definition of interpolation intervals in (13. 2p . The 
value function of the controlled Markov chain is 



Vl^{x,i)= sup J'^{x,i,7r^). (3.5) 

TT^ admissible 

We shall show that V^(x,£) satisfies the dynamic programming equation: 

max ( V e-^^^C-'^'-.i)/ ((x, i), {y, i)\n)V'^{y, l) + /(x, i, u)At\x, i, u, 1), 

11.(^11 * ^ 



0, 



[J2p''ii^J)dy,>^)\^)V''{y,i) + c{x,i)h] y ioTxeSh, 

for X = 0. 

(3.6) 

Note that discounting does not appear in the second line above because singular control is 
impulsive. In the actual computing, we use iteration in value space or iteration in policy 
space together with Gauss-Seidel iteration to solve V'^. The computations wi ll be very 
involved. In contrast to the usual state space Sh in iKushner and Dupuid (120011 ) . here we 
need to deal with an enlarged state space Sh x M. due to the presence of regime switching. 

Define the approximation to the first and the second derivatives of V{-,i) by finite dif- 
ference method in the first part of QVIs (12.41) using stepsize h > a.s: 



VixJ) -> V^{x,i) 

V^{x + h,i) -V''{x,- 



V,{x,i)^ 



V^ixJ)-^ 



V'ix, 



V^ix - h, 



h 



for b{x, i, u) > 0, 
for b{x, i, u) < 0, 



(3.7) 



V^{x + h,i)- 2V^{x, i) + V^{x - h, . 



For the second part of the QVIs, we choose 

V^{x,i) - V^{x-h, 



v.{x,e) ^ 



h 



Together with the boundary conditions, it leads to 
V''{x,£) = 0, for X = 0, 



max 

+ 



Vix + hJ) -V^'{x,. 



-b(x, i, u) 



V'ixJ) - V^{x-h,. 



V'ix + h,e)- 2V^{x, i) + V^{x - h, i) a2(x, i, u) 



-b{x, i, u) 



V^(x, 



V^ix - h, 



+ J2v''i^r)qi.-rV''{x,i)+u, c{x,e 
yxeGlieM, 

where b{x, £, u)~^ and b{x, i, u)~ are the positive and negative parts of 6(x, £, m), respectively. 
Simplifying (13.81) and comparing the result with (13. 6p . we achieve the transition probabilities 
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of the first part of the right side of (13. 6p as the following: 



/((X, i),ix + h, = 



with 



p\(x,i),{x,i)\n) = ^^^qe., for £ ^ ^^'^^ 
p^{-) = 0, otherwise, 



D = (t\x, i, u) + h\b{x, i,u)\ + h\r - 



being well defined. We also find the transition probability for the second part of the right 
side of (13.61) . That is, 

p''i{x,i),ix-h,£)\n) = 1. 

Remark 3.1. The transition probabilities are quite natural. The first part of the QVIs 
can be seen as a "diffusion" region, where the regular control is dominant. The Markov 
approximating chain can switch between regimes and states nearby. But the second part of 
the QVIs is the "jump" region, where the dividends are paid out and the singular control is 
dominant. The singular control will project the Markov approximation chain back one step 
h w.p.l due to the representation. 

Since the wealth can not reach infinity, we need only need choose B large enough and com- 
pute the value function in the finite interval [0, B]. Our ultimate goal is to show converges 



to V i n a large enough interval [0, -B] as h ^ 0. A common approach (IKushner and Dupuis 



(I2OOII )) is to show that the collection {^{^^ , a'^) , , , , h > O} is tight and then appro- 
priately characterize the subsequential weak limit. However, the above scheme is prob- 
lematic since in general, the processes \^g^ {■) , {■) , h > O j may fail to be tight . To over- 



come this difficulty, we ada pt the techniques developed in lKushner and Martins! ( 1l99ll ) and 



Budhiraja and Rosd (120071 ). The basic idea is to (a) suitably re-scale the time so that the 
processes involved in the convergence analysis are tight in the new time scale; (b) carry out 
weak convergence analysis with the rescaled processes; and (c) revert back to the original 
time scale to obtain the convergence of to V. Note that the setting in our problem 
is different from those in the aforementioned references. Moreover, the presence of regime 
switching adds additional difficulty in the analysis. 



4 Convergence of Numerical Approximation 

This section focuses on the asymptotic properties of the approximating Markov chain pro- 
posed in the last section. The main techniques are methods of weak convergence. To begin 
with, the technique of time rescaling is given in Section UT] The interpolation of the approxi- 
mation sequences is introduced in Section H?T1 The definition of relax controls and chattering 
lemmas of optimal control are presented in Sections 14.21 and 14. 3[ respectively. Section 14.41 
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deals with weak convergence of {^^{■),a'^{-),m^{-),w^{-), z^{-), g^{-),T^{-)}, a sequence of 
rescaled process. As a result, a sequence of controlled surplus processes converges to a limit 
surplus process. By using the techniques of inversion, Section 14.41 also takes up the issue of 
the weak convergence of the surplus process. Finally Section H75] establishes the convergence 
of the value function. 

4.1 Interpolation and Rescaling 

Based on the approximating Markov chain constructed above, the piecewise constant in- 
terpolation is obtained and the appropriate interpolation interval level is chosen. Recalling 
(13. 3p . the continuous-time interpolations {^^{■),a^{-)), u^{-), g^{-), and z'^i^-) are defined. In 
addition, let W'^ denote the collection of controls, which are determined by a sequence of 
measurable functions -F^(-) such that 

«n = ^n(e^«tfc<^;«t/^<^). (4.1) 

Define as the smallest cr-algebra generated by {^^{s),a^{s),u'^{s), g^{s), z^{s), s < t}. 
In addition, defined by (14.11) is equivalent to the collection of all piecewise constant 
admissible controls with respect to P^. 

Using the representations of regular control, singular control, reflection step and the 
interpolations defined above, (13.11) yields 

n-1 

e{t) = X + + - EtA^k)] - z\t) - g\t) 

k=0 

n— 1 n— 1 

= ^ + E ^(^'' ^k)^t'i^t «t 4) + J2(Ae, - Ej^Ae,) - z\t) - g\t) + e\t) 
= x + B^it) + M'^it) - z^it) - g^{t) + £^(t), 

(4.2) 

where 

n-1 

B'\t) = J]&(4^«.^«^)A^^e^«,^4), 

fe=0 

n-1 

M'{t) = Y,i^ek-E',Ae,), 

fc=0 

and e'^{t) is a negligible error satisfying 

lim sup E\e''{t)\'^ -¥ for any < T < oo. (4.3) 

/t-s>ooo<t<T 

Also, M^{t) is a martingale with respect to V^, and its discontinuity goes to zero as /i — )■ 0. 
We attempt to represent M^{t) in a form similar to the diffusion term in (12.11) . Define w'^{-) 
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as 

n-l 



k=o (4.4) 
"\-\e{s),a\s),u\s))dM\s). 



We can now rewrite (14 ■2p as 

^''(t)=x+ [ b{^^{s),a^{s),u^{s))ds+ [ a{^''{s),a^{s),u^{s))dw^{s)-z''{t)-g^{t)+e^{t). 
Jo Jo 

(4.5) 

Next we will introduce the rescaling process. The basic idea of rescaling time is to "stretch 
out" the control and state processes so that they are "smoother" and therefore the tightness 
of g^{-) and z^{-) can be proved. Define Atjj by 

{At'* for a diffusion on step n, 
|A2;^| = h for a singular control on step n, (4.6) 
\^9n\ = h for a reflection on step n, 

Define f ^(O by 

n — 1 

f^(t) = 5^At^ = t::, for t6[t::,tti] 

i=0 

Thus, T^(-) will increase with the slope of unity if an only if a regular control is exerted. 
In addition, define the rescaled and interpolated process ^^(t) = C,^{T^{t)), likewise define 
a'^{t)), u^{t), g^it) similarly. The time scale is stretched out by h at the reflection and 
singular control steps. We can now write 



e{t)=x + / h{e{s),a\s),u\s))ds 

(4.7) 







+ / a{C{s),&\s),u''{s))dw\s)-z''{t)-g\t)+e\t). 
Jo 

4.2 Relaxed Controls 

Let B{U X [0, oo)) be the a-algebra of Borel subsets of f/ x [0,oo). An admissible relaxed 
control (or deterministic relaxed control) m(-) is a measure on B{U x [0,oo)) such that 
m{U X [0,t]) = t for each t > 0. Given a relaxed control m(-), there is an mt{ ) such that 
m\d(t)dt) = mt{d(j))dt. We can define mt{B) = lim^^o ^^^^^^^^r^ ^^r B e B{U). With the 
given probability space, we say that m(-) is an admissible relaxed (stochastic) control for 
(w(-), «(■)) or (m(-), w(-), «(■)) is admissible, if m(-, u) is a deterministic relaxed control with 
probability one and if m{A x [0,t]) is J-^-adapted for all A G B{U). There is a derivative 
mt{-) such that mt{-) is J-^-adapted for all A G B{U). 

Given a relaxed control mi-) of u^(-), we define the derivative mt(-) such that 



m^(ir) = / J|(„,.,t)g;^|mt(d0)rft (4.^ 

't/x[0,oo) 
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for all K E B{U x [0,oo)), and that for each t, mt{-) is a measure on B{U) satisfying 
mt{U) = 1. For example, we can define mt{-) in any convenient way for t = and as the 
left-hand derivative for t > 0, 

m,{A) = hm ^^^^ e B{U). (4.9) 

(5^0 

Note that m^dcjxit) = mt{d(f))dt. It is natural to define the relaxed control representation 
m'^(-) of u^(-) by 

mf (A) = \/A G B(U). (4.10) 

Let J-'^ be a filtration, which denotes the minimal a-algebra that measures 

{e{s),a\-),m%),w\s),z\s),g\s),s<t}. (4.11) 

Use to denote the set of admissible relaxed controls m^(-) with respect to (a^(-), w'^(-)) 
such that mJl^-) is a fixed probability measure in the interval [tjj,t^_|_i) given J^f . Then 
is a larger control space containing U^. Referring to the stretched out time scale, we denote 
the rescaled relax control as mfh^^^-^^dcj)). Define Mt{A) and M^^dcj)) by 

Mt{A)dt = dw{t)Iu{t)eA, VA G B{U) 
M^{d(j))dt = dw\t)I^,(^t)eu- 

Analogously, as an extension of time rescaling, we let 

M|,(,)(rf0)rff ^(t) = rf^^(r'^(t))j„.(^.(,)),^. 

With the notation of relaxed control given above, we can write (14.51) . (14. 7p and the value 
function (12. 3p as 

^f^(^t) = x+ [ [ b{^''{s),a''{s),(j))m'^{d(f))ds+ [ [ ^(^^(s), a'^(s), 0)M,^(#)rfs + £^(t), 
Jo Ju Jo Ju 

(4.12) 
(4.13) 



t{t) = x + J J^bii\s),a\s),<l>)mh^^^^idcj))df\s) 

+ [ [ a(e^.),d^s),0)M^.(,)(rf0)rfr'^(s)-i'^(t)-^^t)+e^t), 



and 

V''{x,i)= inf J'^(x,£,m'^). (4.14) 

Now we give the definition of existence and uniqueness of weak solution. 

Definition 4.1. By a weak solution of (14.121) . we mean that there exists a probability space 
{Q,J-',P), a filtration J't, and process (x(-), a(-), m(-), w(-)) such that w{-) is a standard 
J-^- Wiener process, a{-) is a Markov chain with generator Q and state space Ai, m(-) is 
admissible with respect to x(-), and is J-'r adapted, and (14.121) is satisfied. For an initial con- 
dition (x, i), by the weak sense uniqueness, we mean that the probability law of the admissible 
process (a(-), m(-), w(-)) determines the probability law of solution (x(-), «(■), m(-), w(-)) to 
( I4.12p . irrespective of probability space. 
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To proceed, we need some assumptions. 



(Al) Let u{-) be an admissible ordinary control with respect to w{-) and a(-), and suppose 
that m(-) is piecewise constant and takes only a finite number of values. For each 
initial condition, there exists a solution to fl4.12p where m(-) is the relaxed control 
representation of u{-). This solution is unique in the weak sense. 



4.3 A Chattering Lemma and Approximation to the Optimal Con- 
trol 

This section deals with the approximation of relaxed controls by ordinary controls. We 
can always use relax controls to approximate the ordinary controls, which is only a tool for 
mathematical analysis. Here we present a resu lt of chat t ering lemma for our problem. The 
proof of the chattering lemma can be found in iKushner 



Proposition 4.2. Let {m{-),w{-)) be admissible for the problem given in (14.121) . Then 
given q > 0, there is a finite set {7^, . . . ,7^^^} = U'' C U, and an e > such that there is 
a probability space on which are defined {x'^{-),a'^{-),u'^{-),w^{-)), where w'^{-) are standard 
Brownian motions, and u''{-) is an admissible If'-valued ordinary control on the interval 
[ke, ke + e). Moreover, 



im 



P. 



( sup \x''{s) — x{s)\ > I < ^, and 

\s<T J 



(4.15) 



i4"^(-)-^r(-)i<'^- 



Coming back to the approximation to the optimal control, to show that the discrete 
approximation of the value function V'^{x,i) converges to the value function V{x,i), we 
shall use the comparison control techniques. In doing so, we need to verify certain continuity 
properties. The details of the proof is presented in the appendix. 

Proposition 4.3. For (14.121) . let^> be given and (x(-), a(-), m(-), w(-)) be an ^-optimal 
control. For each q > 0, there is an e > and a probability space on which are defined w'^{-), 
a control m^(-) as in Theorem 14. 2 j. and a solution x''{-) such that the following assertions 
hold: 



(i) 



\JTi-)-j:\-)\<'^- (4.16) 

Moreover, there is a 6 > such that the approximating «''(■) can be chosen so that 
its probability law at ne, conditioned on {w'' (t) , a'' (t) , t < ne;u''{ke),k < n} depends 
only on the samples {w'' {p6) , a'' {p6) , p6 < ne]u'^{ke), k < n}, and is continuous in the 
w'^{p9) arguments. 
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4.4 Convergence of A Sequence of Surplus Processes 



Lemma 4.4. Using the transition probabilities {p^{-)} defined in ( \3.9\\ . the interpolated 
process of the constructed Markov chain {«^(-)} converges weakly to a{-), the Markov chain 
with generator Q = (g^). 



P roof. It can be s een that (y^{-) is tight. The proof can be obtained similar to Theorem 3.1 



m 



Yin et all ((20031). That is, 



\E[{a''{t + s)-a''{t)f]\TM<^{s) and hm hm sup ^7(5) = 0, (4.17) 

where 7(5) > is J-'/'-measurable. On the other hand, due to the definition of we have 

\E[ia\t + s) - a\t)nj^,'\ < \E[{a\t + s)-a\t)nj^,'\<^{s). (4.18) 

Combining (jUTTj) and (14.181) . we obtain a^{-) is tight. Thus, the constructed Markov chain 
{d^(-)} converges weakly to «(■). □ 

Theorem 4.5. Let the approximating chain {^^,a;^,n < 00} constructed with transition 
probabilities defined in (13.91) be locally consistent with (12. ip . m^{-) be the relaxed control 
representation of {u^,n < 00}, (^^(■), a^(-)) be the continuous-time interpolation defined 
in (13.31) . and {^'^{■),a^{-),m'^{-),w^{-), z^{-), g^{-),T^{-)} be the corresponding rescaled pro- 
cesses. Then {i''{-),a^{-), m''{-),w^{-),z'' {■),g^{-),f ''{■)} is tight. 



Proof. In view of Lemma 14.41 is tight. The sequence {m^{-)} is tight since its range 

space is compact. Let T < 00, and let be an J-'^-stopping time which is no bigger than T. 
Then for 6 > 0, 

Ef^{w\Tn + 6)- w\Th)f = 5 + (4.19) 

where — > uniformly in r^. Taking limsup^_^o followed by lim^^o yield the tightness of 
{w^{-)}. Similar to the argument of a''(-); the tightness of w^{-) is obtained. Furthermore, 
following the definition of "stretched out" timescale, 

\z\Th + 5)~z\Th)\ < |5|+0(/i), 
\g\rH + 5) - g\rh)\ < \5\+0{h). 

Thus {z^{-),g^{-)} is tight. For notational simplicity, we assume that 6(-) and a{-) are 
bounded. For more general case, we can use a truncation device. These results and the 
boundedness of b{-) implies the tightness of Therefore it follows that 

{e(-),«'(-),«'(-),^'(-),^'(-),^'(-),T^-)} 

is tight. □ 
Since {x'^{-),&^{-),m^{-),w^{-),z^{-),g^{-),T^{-)} is tight, we can extract a weakly con- 
vergent subsequence denoted by {^(■), d;(-), m(-), t&(-), z(-), 5f(-), T(-)}. Also, the paths of 
{x(-), «(■), m(-), w{-), z(-), 5f(-), T(-)} are continuous w.p.l. 
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Theorem 4.6. Let {x{-),a{-),m{-),w{-), z{-), g{-),T{-)} be the limit of weakly convergent 
subsequence of {^^ {■) , {■) , {■) , {■) , z'^ {■) , g'^ {■) , {■)} . w{-) is a standard J-'t- Wiener 
process, and m{-) is admissible. Let J^t the a-algebra generated by {,^^(-), Q!''(-), m''(-), 
z^[-),g^{-),T^[-)}. Then w(t) = w{T(t)) is an fit-smarting ale with quadratic variation T{t). 
The limit processes satisfy 



x{t)=x + I I b{x{s),a{s),(f))m!p^^^{d(f))dT{s 



Ju 



JU 



(4.20) 

a{x{s),a{s),<j))Mf^^){d<f))df{s) - z{t) - 9{t)- 



Proof. For 5 > 0, define the process /(■) by l^'^{t) = l^{n5),t e [nS, {n + 1)5). Then, by the 
tightness of {^^{■),a^{-)}, (14.131) can be rewritten as 

i\t)=x+ f [ b{i\s),a\s)A)m% M)df\s) 

Ju (4-21) 
+ 11 cTii'^'\s),a''\s),<P)Mf,^,^{d<p)df\s)-z\t^^ 



JU 



where 



\im\imsup E\e^'\t)\ = 0. (4.22) 



If we can verify w(-) to be an .Ff-martingale, then ( I4.20p could be obtained by taking 
hmits in (I4.2ip . To characterize w{-), let t > 0, 5 > 0, p, q, {tk : k < p} be given such 
that tfc < ^ < ^ + s for all < p, ipj{-) for j < g is real-valued and continuous functions on 
U X [0, oo) having compact support for all j < q. Define 

(V^„m)i = ^ jj,{(t),s)m%^^^{d<P)df{s). (4.23) 

Let S{-) be a real-valued and continuous function of its arguments with compact support. 
By (14. 4p . w^{-) is an J-'^-martingale. In view of the definition of ti)(t), we have 

ES{i\tk). a\tk), w\tk), (^„ m%^,z\tk),g\tk),j <q,k< p)[w\t + s) - w\t)] = 0. 

(4.24) 

By using the Skorohod representation and the dominant convergence theorem, letting /i — )■ 0, 
we obtain 

ES{i''{tk),a''{tk),w''{tk)d^3,rn%,z''{tk),g''{tk),j <q,k<p)[w^^ (4.25) 

Since w{-) has continuous sample paths, (I4.25P implies that t&(-) is a continuous J^^-niartingale. 
On the other hand, since 

Eliiw^'it + 6)f - {w''{t)f] = E[{w'\t + 5) - w''{t)f] = f{t + s)- f{t), (4.26) 

by using the Skorohod representation and the dominant convergence theorem together with 
(H^ni), we have 



ES{i\tk), a\tk),w\t,), (^„ m%^,z\tk),g\tk),j <q,k<p) 
X [w\t + 6)- w\t) - {f{t + s)- f{t))] = 0. 
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The quadratic variation of the martingale w{t) is AT, then w{-) is an J^- Wiener process. 
Let /i — )■ 0, by using the Skorohod representation, we obtain 



E 



h{i\s),a\s),<P)m'l Ad<P)df\s)- f / h{x{s),a{s),<P)m'l {d<P)dT{s) 

Jo Ju 

(4.28) 

uniformly in t with probability one. On the other hand, {m'^(-)} converges in the compact 
weak topology, that is, for any bounded and continuous function ip{-) with compact support, 
as /i — )• 0, 







oo r roo 



^{(t^,s)mh {dcj>)dT\s)^ / ^{<p,s)mf^^^{d<P)dT{s). (4.29) 

IQ JU ^ ' Jo Ju 

Again, the Skorohod representation (with a slight abuse of notation) implies that as /;,—)■ 0, 
ft r rt 



h{e{s),a\s),<P)m% M)dT\s)^ / h{x{s),a{s),<P)mf,Ad<P)df{s^ (4.30) 
JU Jo Ju 

uniformly in t with probability one on any bounded interval. 

In view of fl4.2ip . since and a^''^(-) are piecewise constant functions, 

a{t^\s),a^^\s),(t))Mfn^^^[d(j))df^{s)^ [ [ a{x\s),a\s),<P)Mf(^^)id<f>)df{s) 
'0 JU Jo Ju 

(4.31) 

as /i -> 0. Combining fl4.23p -( g3T]) . we have 



x{t) = X + I I b{x{s),a{s),(f))m^, Jd(f))dT{s) 

" (4.32) 
cr{x\s), a\s), ^)Mf^^){d(P)df{s) - z{t) - g{t) + e\t), 

'0 JU 

where lims^Q E\e^ (t)\ = 0. Finally, taking limits in the above equation as 5 — )■ 0, fl4.20p is 
obtained. □ 

Theorem 4.7. For t < oo, define the inverse 

R{t) = inf{s : f{s) > t}. 

Then R{t) is right continuous and R{t) oo as t ^ oo w.p.l. For any process define 
the rescaled process by (f{t) = (p{T{t)). Then, w{-) is a standard J^t-Wiener process and 
(IXTj) holds. 

Proof. Since T{t) — )■ oo w.p.l as t — )■ oo, R{t) exists for all t and R{t) — )■ oo as t — )■ oo 
w.p.l. Similar to and (g^ZD, 

E5(e^tfc),«^tfc),w;^tfc),(V^j,m^)t,,2^(4),/(4),j <g,A;<p) x \w{t + s) ~ w{t)] = 

ESieitk), a^4), ^"(4), iiJ,, m%^,z\tk),g\tk),j <q,k<p) 
X [w^{t + 6)- w'^it) - {R{t + s)- R{t))] = 0. 
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Thus, we can verify w{-) is an J-^- Wiener process. A rescaling of fl4.20p yields 

x{t) = X + / / b{x{s), a{s), (j))ms{d(j))ds 
Jo^ Ju 

+ 11 a{x{s),a{s),(f))Ms{d(j))ds - z{t) - g{t). 



(4.33) 

7{x{s), a{s), (f))Ms{d(t>)ds — z{t) — g(t). 
Jo Ju 

In other words, (12. ip holds. □ 
4.5 Convergence of Cost and Value Functions 

Theorem 4.8. Let h index the weak convergent subsequence of {^^{■),a'^{-),m^{-),w^{-), 
z''{-),g^{-),f with the limit {x{-),a{-),m{-),w{-),z{-),g{-),f{-)}. Then, 

J''{x,i,n^) ^ [ [ e-'^^'\f{x{t),a{t),(f))mt{d(f))dt + c{x{t-),a{t-))dZ] 

= / / e~'^*[/(x(t), a{t), (f))mt{d(j))dt + c(x(r ), a{t-))dZ] = J{x, £, vr). 
Jo Ju 

(4.34) 

Proof. Note that Az^ = Ag^ = h, the uniform integrability of dZ can be easily verified. 
Due to the tightness and the uniform integrability properties, for any t, 

t 

c{x{t^), a{t^))dZ 



can be well approximated by a Reimann sum uniformly in h. By the weak convergence and 
the Skorohod representation, 

4(x, ^, 7^^)=eY^ e-< [/(eL u\)At\ + c(eL al)Azl\ 

k=l 

^El^f [ e^''^^''^[f{x{t),a{t),(P)mtid(P)dt + c{x{t'),a{t-))dZ]. 
Jo Ju 

By an inverse transformation, 

E^e f f e-'^^'\f{x{t),a{t),(f))7htid(f))dt + c{x{t-),a{t-))dZ] 



Jo JU 



= El^ / / e~''*[/(x(t), a{t), (j))mt{d(j))dt + c(x(r), a{t-))dZ]. 
Jo Ju 

Thus, as /i — )• 0, 

J^{x,l,Ti^) J(x,£,7r). 



□ 



Theorem 4.9. V^{x^t) andV{x,i) are value functions defined in (14.141) and (12. 3p . respec- 
tively. Then V^{x,i) ^V{x,£) ash^O. 
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Proof. First, to prove 

\imsupV''{x,i) < V{x,i). (4.35) 

h 

Since V{x,i) is the maximizing cost function, for any admissible control vr(-), 

j{x,e,TT) < v{x,e). 

Let rhP'{-) be an optimal relaxed control for {^^{■)} and 7r'^(-) = {rhP'{-),^^{-)^'g^{-)). That is, 

V^{x, i) = J^{x, i, n^') = sup f\x, i, tt'^). 

Choose a subsequence {h} of {h} such that 

lim V~^{x, i) = limsup V~^{x, I) = lim /(x, £, 

h^O 

Without loss of generahty (passing to an additional subsequence if needed), we may assume 
that a^{-),m^{-),w^{-), z^{-),g^{-)) converges weakly to (x(-), a(-), m(-), w(-), z{-),g{-)), 

where 7r(-) is an admissible related control. Then the weak convergence and the Skorohod 
representation yield that 

limsup V^{x, e) = J(x, i, vr) < V{x, i). (4.36) 

h 

We proceed to prove the reverse inequahty. 
We claim that 

liminf K'^(a;,£) > V{x,i). (4.37) 

h 

Suppose that m is an optimal control with Brownian motion w{-) such that x(-) is the 
associated trajectory. By the chattering lemma, given any 7 > 0, there are an e > and 
an ordinary control u'^{-) that takes only finite many values, that u'~'{-) is a constant on 
[ke,ke + e), that nP^-) is its relaxed control representation, that (ar'{-),f?P{-)) converges 
weakly to (x(-),m(-)), and that J{x,i,Tf) > V{x,i) —7. 

For each 7 > 0, and the corresponding e > as in the chattering lemma, consider 
an optimal control problem as in (12. ip with piecewise constant on [ke,ke + e). For this 
controlled diffusion process, we consider its 7-skeleton. By that we mean we consider 
the process {x'' (ke) , m'^ (ke)) . Let S^(-) be the optimal control, m'^{-) the relaxed con- 
trol representation, and x^{-) the associated trajectory. Since m"'{-) is optimal control, 
J{x, i, w?) > J{x, i,fn'^) > V{x, i) — 7. We next approximate m^(-) by a suitable function of 
(«;(■),«(■))• Moreover, V^'ixJ) > J^{x, i,m'') J{x,i,frP'^) Thus, 

liminf \/'^(x,£) > J^{x,l,m^) J(x, £, rrT'^). 

h 

Using the result obtained in Proposition 14. 3[ 

lim inf V^{x, £) > V{x, £) - 27. 

h 

The arbitrariness of 7 then implies that liminf/i V"'*(a;, £) > V{x,i). 

Using f l4.36p and f l4.37p together with the weak convergence and the Skorohod represen- 
tation, we obtain the desired result. The proof of the theorem is concluded. □ 
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5 Numerical Example 



This section is devoted to a couple of examples. For simplicity, we consider the case that the 
discrete event has two states. That is, the continuous-time Markov chain has two states. We 
approximate the value functions in the case of the claim size distributions are given. Pro- 
portional reinsurance and nonproportional reinsur ance are consid e red, r espectively. These 
results are compared to the numerical examples in lAsmusen et al.l (120001) ■ 



5.1 Proportional Reinsurance 

Example 5.1. The generator of the Markov chain a{t) is 



Q 



-0.5 0.5 
0.5 -0.5 



and Ai = {1, 2}. The claim rate depends on the discrete state with /3(1) = 1 and /3(2) = 10. 
Assume the claim size distribution to be exponential with parameter 1. Then E[Y] = 1 and 
E[Y^] = 2. The dllD follows 



dX{t) = f3{a{t))u{t)dt + ^/2f3{a{t))u{t)dw{t) - dZ{t), 
X(O-) = X 

where the retention level u{t) is the regular control parameter representing the fraction of 
the claim covered by the cedent and u(t) G [0,1]. Taking the discount rate r = 0.05, we 
compare the cost function in the case of the total expected discounted value of all dividends 
until lifetime ruin mentioned in (11. 6p 



J{x,e,7r) = E,,, [ e-'-'dZit). 



[0/ 

and the differential marginal yield to m easure the inst antaneous returns accrued from irre- 



versibly exerting the singular policy, see lAlvarezI (120001 ). 



J{x, i, tt) = E^,e I Ae-"*-^^(^)dZ(t), 

J\0.t) 



'[0,r) 

where A = 1. We obtained Figure [5?2l for this case. 

Example 5.2. In this example, the claim size distribution is assumed to be uniform in [0, 1]. 
Then = | and E'fY'^] = |. Then the dynamic systems follows 

{dX{t) = l(3{a{t))u{t)dt + ^\(5{a{t))u{t)dw{t) - dZ{t), 
\x(0-) = X. 

Using the same data and payoff function in Example 15. H we then obtained Figure 15.21 
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■ a = 1 
•a = 2 



Initial Surplus 

5.1.1 Total expected discounted value of all divi- 
dends versus initial surplus 



a 
o 

2.5 



■ a = 1 
■a = 2 



Initial Surplus 

5.1.2 Differential marginal yield versus initial sur- 
plus 




■ Q = 1 
■Q = 2 



Initial Surplus 



0.35 
>, 0.3 

° 0.25 

ID 
U 

e 0.2 



Pi 0.- 



Q = 1 
— Q = 2 



Initial Surplus 



5.1.3 Optimal reinsurance policy to total expected 5.1.4 Optimal reinsurance policy to differential 

discounted value of all dividends versus initial sur- marginal yield versus initial surplus 

plus 



Figure 5.1: Proportional reinsurance with exponential claim size distribution with two 
regimes 
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■ a = 1 
■a = 2 



Initial Suiplus 

5.2.1 Total expected discounted value of all divi- 
dends versus initial surplus 



if 



u 



"■ o = 1 
— Q = 2 



Initial Suiplus 

5.2.2 Differential marginal yield versus initial sur- 
plus 



o 

0.6 

tu 
u 

C 0.5 
S 0.4 

.g 

"u 0.3 



■ a = 1 
■Q = 2 



Initial Surplus 



o 

0- o.f 



C 0.5 

3 0.4 



OJ 0.3 

OS 



-0=1 
— Q = 2 



Initial Surplus 



5.2.3 Optimal reinsurance policy to total expected 5.2.4 Optimal reinsurance policy to differential 

discounted value of all dividends versus initial sur- marginal yield versus initial surplus 

plus 



Figure 5.2: Proportional reinsurance with uniform claim size distribution with two regimes 
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5.2 Excess-of-Loss Reinsurance 



Example 5.3. Comparing to Example 15. H and Example 15. 21 the retention level u{t) describes 
the maximal amount paid by the cedent for each claim. Assume the claim size distribution 
to be exponential with parameter 1 and Q, /3(l),/3(2) and payoff functions to be the same 
as those in Example 5.1. Intuitively, the retention level cannot be arbitrarily large, then we 
restrict the risk control set U to be [0, 1]. That is, the retention level should not exceed the 
mean value of the exponential distributed claim size. Following (11. 3p 

= / e-'^dx = 1 - e-"", 

E[(F")2]= / 2xe-"(ix = 2[l-e-"(l + M)]. 
Jo 

Then the dynamic systems satisfy 



dX{t) = /3{a{t))[l - e~"(*)]cit + ^2/3(a(t))[l - e-«W(l + u{t))]dw{t) - dZ{t), 
X(0~) =x; 

see Figure 15731 for this case. 

Example 5.4. Assume the claim size distribution to be uniform in [0,1]. Similarly, we 
obtain 



E[ir 



u 



M\2l 



(1 — x)dx = u — — 



2x(l — x)dx = 



1 _ 2" 



Hence, the dynamic systems satisfy 



dX{t) 

x(o-) 



dt 



P{a{t))u{ty 



2u{t) 



dw{t) - dZ{t), 



X. 



Let the risk control set U = [0, 1]. We obtain Figure [531 in this case. 



All of the figures contain two lines since we consider the two-regime case. Figures 15.1.11 
15.2.11 l53Tn and 15^4. II show that the value function is concave and the dividend payout strategy 
is a barrier strategy. It is clear that if the surplus is higher than some barrier level, the extra 
surplus will be paid as the dividend, with the same time the value functions increase with 
unity slope. 

Regarding the reinsurance policy, it is demonstrated in Figures I5.1.3[ I5.2.3[ I5.3.3[ and 
15.4.31 that both the proportional reinsurance and excess-of-loss reinsurance increase at first, 
maintain the highest rate in an interval, and decrease sharply to zero at a threshold to 
maximize the total expected discounted value of all dividends. From Figures I5.1.4[ I5.2.4[ 
I5.3.4[ I5.4.4[ in the case of maximizing the differential marginal yield, it is shown that both 
the proportional reinsurance and excess-of-loss reinsurance have similar trend comparing to 
the case of maximizing the total expected discounted value of all dividends, except that 
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■ a = 1 
■a = 2 



Initial Surplus 

5.3.1 Total expected discounted value of all divi- 
dends versus initial surplus 



§4 



u 



■ a = 1 
■a = 2 



Initial Surplus 

5.3.2 Differential marginal yield versus initial sur- 
plus 




Initial Surplus Initial Surplus 

5.3.3 Optimal reinsurance policy to total expected 5.3.4 Optimal reinsurance policy to differential 

discounted value of all dividends versus initial sur- marginal yield versus initial surplus 

plus 

Figure 5.3: Excess-of-loss reinsurance with exponential claim size distribution with two 
regimes 
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Figure 5.4: Excess-of-loss reinsurance with uniform claim size distribution with two regimes 
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there are not the interval to hold the highest reinsurance rate. Furthermore, it is shown 
that in both regimes, there exists a free boundary (barrier) that separates two regions where 
the regular control or singular control is dominant. Also, the barrier levels are different in 
different regimes due to the Markov switching. 

In addition, we compare the values for proportional reinsurance and excess-of-loss rein- 
surance with exponential claim size distribution in Table 15.11 At the level of initial surplus 
X = 30, we compare the corresponding values in two regimes. Similarly we have the com- 
parison in Table 15.21 for the uniform claim size distribution. 



Reinsurance type 


a = 1 


a = 2 


proportional reinsurance 


127.661229 


136.139963 


excess-of-loss reinsurance 


128.207117 


136.686110 



Table 5.1: ^(30, a) with exponential claim size distribution for proportional reinsurance and 
excess-of-loss reinsurance 



Reinsurance type 


a = 1 


a = 2 


proportional reinsurance 


79.010314 


83.256482 


excess-of-loss reinsurance 


80.097716 


84.302264 



Table 5.2: V^(30, a) with uniform claim size distribution for proportional reinsurance and 
excess-of-loss reinsurance 



From Tables 15.11 and 15.21 we see that V(30, a) of excess-of-loss reinsurance are bigger 
than that of proportional reinsurance in both of the two regimes. That is, we can conclude 
that the excess-of-loss reinsurance is more profitable than proporti onal reinsuranc e unde r 
the same condition. This is consistent with the one regime case in lAsmusen et al.l ( 120001 ). 
Finally, the numerical method can treat complicate cost functions such as the marginal yield, 
which is another advantage of the numerical solutions. 



6 Further Remark 

In this work, we have developed a numerical approximation scheme to maximize the payoff 
function of the total discounted dividend paid out until the lifetime of ruin. A generalized 
formulation of reinsurance and dividend pay-out strategy is presented. Although one could 
derive the associated system of QVIs by using the usual dynamic programming approach 
together with the use of properties of regime- switchings, solving for the mixed regular- 
singular control problem analytically is very difficult. As an alternative, we presented a 
Markov chain approximation method using mainly probabilistic methods. For the singular 
control part, a technique of time rescaling is used. In the actual computation, the optimal 
value function can be obtained by using the value or policy iteration methods. Examples 
of proportional and excess-of-loss reinsurance are presented with more complicated payoff 
functions. 
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A Appendix: Proof of Proposition 113 



The proof is similar in spirit to Yin et aP fcoooh however regime - switch ing; is included. The 



technique are originated from the work of iKushner and Dupuid (120011 ). For simplicity, We 



divide the proofs into several steps. First, for any <^ > 0, by Theorem 14. 2 [ there are e > 0, a 
finite set U'' C U, and a probability space on which are defined a solution in the stochastic 
differential equation in (12. ip . Thus, we have (a;'^(-), a'^(-), «''(■), w'^(-)), where f/'^(-) is f/"^- 
valued and constant on [ne,ne + e). Moreover, {x"" {■) , a"" {■) , m'- {■) , w'' {■)) converges weakly to 
(X(-), a(-), m(-), the solution of the differential equation in (12. ip . This further implies 

that limsup^ \JTli') ~ ^ with satisfying — > as ?'— 0. 

Next, consider a «''(■) for q sufficiently small. Let < < e. For G t/'^, define the 
function Fn^e as the regular conditional probability 

Fn,e{4>yW'{ie),i < n,w'"{p9),p9 < ne) = P{u'{ne) = (j)\u''{ie),i < n,w'"{p9),p9 < ne). 

The uniqueness of the solution of the wealth equation or the associated martingale problem 
implies that the law of (x'', a'^(-), m'^(-), ty'^(-)) is determined by the law of (a^(-), m''(-), 
since the cr-algebra determined by {u'^{ie)^i < n,w''{p9),a''{p9),p9 < ne} increases to the 
cr-algebra determined by {u''{ie),i < n, w'-^t), a''(r), r < ne} as 6* — 0, we can show that for 
each n, 0, and e, Fnfi{(t),u'^{ie),i < n,w''{p9),a''{p9),p9 < ne) — t- P{u'^{ne) = (f)\u'' (ie) , i < 
n, w''{t), a'^(r), r < ne) with probability one as — ?■ 0. 

For w'^'^{-), define the control m'^'^(-) by the conditional probability given in F„ ^ with <^ re- 
placed by 9. Owing to the construction of the control law, as 6* — )■ 0, (a'^'^(-), m'>'^(-), ly'^'^(-)) 
converges weakly to («'■(■), m''(-), We can further show {x'''^ {■) , a""^ {■) , m'-'^ {■) , w'''^ {■)) 

converges weakly to {x'^{-), a"^ {■) , m"^ {■) , w'' {■)) as — )■ 0, and moreover x^'^{-) converges weakly 
to x'{-). Thus IJ™"'!-) - J'S^li-)\ < 9ii&), where ^i(^) ^ as ^ 0. 

For A > 0, consider the mollifier -F„,e,A(') given by 

FnfiA{4>'i'u{ie),i < n,w{p9),p9 < ne) 

= N{A) /■•■ / Fn^9{4>;u{ie),i < n,w{p9) + Zp,p9 <ne) xY\_exp{-\zp\'^/{2A))dzp, 

where A^(A) is a normalizing constant so the integral of the mollifier is unity. Note that -F„,e,A 
are nonnegative, and they are continuous in the w-variable. As A — )■ 0, -Fn,e,A converges to 
Fn,0 with probability one. Let m'^'^'^(-) be the piecewise constant admissible control that is 
determined by the conditional probability distribution -F„_e a(-)- There is a probability space 
on which we can define w'^'^'^(-) and the control law by the conditional probability 



P{u''^'^{ne) = (f)\u''^'^{ie),i< n,w''^'^{T),T < ne) 



:u'^'^{ie),i< n,w''^'^{p9),p9 < ne). 



Then a'^'^'^(-), m«'^''^(-), converges weakly to (x^'^(-), a^'^ m'^'^(-), w'^'^(-)) 

as A ^ 0. This yields that {J^','"'^ {■) - < t/2(A), where ^2(A) ^ as A ^ 0. 

Finally, choose ?^ sufficiently small. Then for each <^ > 0, there are e > 0, ^ > 0, w''{-), 
and an admissible control that is piecewise constant on [ne, ne + e) taking values in a finite 
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set U"^ C U determined by the conditional probability law 



Piu'ine) = (f)\u'^ {is) , i < n, w'^{t), a''(r), r < ne) 
= -F„(0; u'{ie),i < n,w'^{pd),a^{pd),p9 < ne) 



where Fn{-) are continuous w.p.l in the w-variables for each of other variables. Moreover, 
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